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jH , We consider optimal control of a multi-class queue in the Halfin- 

Whitt regime, and revisit the notion of asymptotic optimality and 
the associated optimality gaps. The existing results in the literature 
for such systems provide asymptotically optimal controls with opti- 
mality gaps of 0(^^71) where n is the system size, for example, the 
number of servers. We construct a sequence of asymptotically op- 
Ql ' timal controls where the optimality gap grows logarithmically with 

CLj ' the system size. Our analysis relies on a sequence of Brownian con- 

trol problems, whose refined structure helps us achieve the improved 
optimality gaps. 



1. Introduction. Queueing models with many-servers are prevalent in 
modeling call centers and other large-scale service systems. They are used 
^ I for optimizing staffing and making dynamic control decisions. The complex- 

es ' ity of the underlying queueing model renders such optimization problems 

^^ . intractable for exact analysis, and one needs to resort to approximations. 

A prominent mode of approximate analysis is to study such systems in the 
so-called Halfin-Whitt (HW) heavy-traffic regime; cf. [8]. Roughly speaking, 
(3 . the analysis of a queueing system in the HW regime proceeds by scaling up 

CN I the number of servers and the arrival rate of customers in such a way that 

the system load approaches one asymptotically. To be more specific, instead 
of considering a single system, one considers a sequence of (closely related) 
^ I queueing systems indexed by a parameter n along which the arrival rates 

H ' and the number of servers scale up so that the system traffic intensity p" 

- - - satisfies 

(1) ^/n{l-p^)^/3 asn->oo. 
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2 B. ATA AND I. GURVICH 

In the context of dynamic control, passing to a formal limit of the (prop- 
erly scaled) system dynamics equations as n — )• oo gives rise to a limit dif- 
fusion control problem, which is often more tractable than the original dy- 
namic control problem it approximates. The approximating diffusion control 
problem typically provides useful structural insights and guides the design 
of good policies for the original system. Once a candidate policy is pro- 
posed for the original problem of interest, its asymptotic performance can 
be studied in the HW regime. The ultimate goal is to establish that the 
proposed policy performs well. To this end, a useful criterion is the notion 
of asymptotic optimality, which provides assurance that the optimality gap 
associated with the proposed policy vanishes asymptotically under dijfusion 
scaling as n — )■ oo. Hence, asymptotic optimality in this context is equivalent 
to showing that the optimality gap is o{y/n). 

A central reference for our purposes is the recent paper by Atar, Man- 
delbaum and Reiman [2], where the authors apply all steps of the above 
scheme to the important class of problems of dynamically scheduling a mul- 
ticlass queue with many identical servers in the HW regime. Specifically, [2] 
considers a sequence of systems indexed by the number of servers n, where 
the number of servers and the arrival rates of the various customer classes 
increase with n such that the heavy-traffic condition holds; cf. equation (1). 
Following the scheme described above, the authors derive an approximate 
diffusion control problem through a formal limiting argument. They then 
show that the diffusion control problem admits an optimal Markov pol- 
icy, and that the corresponding HJB equation (a semilinear elliptic PDE) 
has a unique classical solution. Using the Markov control policy and the 
HJB equation, the authors propose scheduling control policies for the origi- 
nal (sequence of) queueing systems of interest. Finally, they prove that the 
proposed sequence of policies is asymptotically optimal under diffusion scal- 
ing. Namely, the optimality gap of the proposed policy for the nth system 
is o{^/n). A similar approach is applied to more general networks in [1]. In 
this paper, we study a similar queueing system (see Section 2). Our goal, 
however, is to provide an improved optimality gap which, in turn, requires 
a substantially different scheme than the one alluded to above. 

Approximations in the HW regime for performance analysis have been 
used extensively for the study of fixed policies. Given a particular policy, it 
may often be difficult to calculate various performance measures in the orig- 
inal queueing system. Fortunately, the corresponding approximations in the 
HW regime are often more tractable. The machinery of strong approxima- 
tions (cf. Csorgo and Horvath [4]) often plays a central role in such analysis. 
In the context of many-server heavy-traffic analysis, with strong approxima- 
tions, the arrival and service processes (under suitable assumptions on the 
inter-arrival and service times) can be approximated by a diffusion process 
so that the approximation error on finite intervals is O(logn) (where n is 
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the number of servers as before). Therefore, it is natural to expect that, 
under a given pohcy, the error in the diffusion approximations of the various 
performance metrics is O(logn), which is indeed verified for various settings 
in the hterature (see, e.g., [11]). 

A natural question is then whether one can go beyond the analysis of fixed 
policies and achieve an optimality gap that is logarithmic in n also under 
dynamic control, improving upon the usual optimality gap of o{y/n). More 
specifically, can one propose a sequence of policies (one for each system in the 
sequence) where the optimality gap for the policy (associated with the nth 
system) is logarithmic in nl While one hopes to get logarithmic optimality 
gaps as suggested by strong approximations, it is not a priori clear if this can 
be achieved under dynamic control. The purpose of this paper is to provide 
a resolution to this question. Namely, we study whether one can establish 
such a strong notion of asymptotic optimality and if so, then how should 
one go about constructing policies which are asymptotically optimal in this 
stronger sense. 

Our results show that such strengthened bounds on optimality gaps can 
be attained. Specifically, we construct a sequence of asymptotically optimal 
policies, where the optimality gap is logarithmic in n. Our analysis reveals 
that identifying (a sequence of) candidate policies requires a new approach. 
To be specific, we advance a sequence of diffusion control problems (as op- 
posed to just one) where the diffusion coefficient in each system depends 
on the state and the control. This is contrary to the existing work on the 
asymptotic analysis of queueing systems in the HW regime. In that stream 
of literature, the diffusion coefficient is typically a (deterministic) constant. 
Indeed, Borkar [3] views the constant diffusion coefficient as a characterizing 
feature of the problems stemming from the heavy-traffic approximations in 
the HW regime. Interestingly, it is essential in our work to have the diffusion 
coefficient depend on the state and the control for achieving the logarithmic 
optimality gap. In essence, incorporating the impact of control on the diffu- 
sion coefficient allows us to track the policy performance in a more refined 
manner. 

While the novelty of having the diffusion coefficient depend on the control 
facilitates better system performance, it also leads to a more complex dif- 
fusion control problem. In particular, the associated HJB equation is fully 
nonlinear; it is also nonsmooth under a linear holding cost structure. In 
what follows, we show that each of the HJB equations in the sequence has 
a unique smooth solution on bounded domains and that each of the diffu- 
sion control problems (when considered up to a stopping time) admits an 
optimal Markov control policy. Interpreting this solution appropriately in 
the context of the original problem gives rise to a policy under which the 
optimality gap is logarithmic in n. As in the performance analysis of fixed 
policies, strong approximations will be used in the last step, where we pro- 
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pose a sequence of controls for the original queueing systems, and show that 
we achieve the desired performance. However, it is important to note that 
strong approximation results alone are not sufficient for our results. Rather, 
for the improved optimality gaps we need the refined properties of the solu- 
tions to the HJB equations. Specifically, gradient estimates for the sequence 
of solutions to the HJB equations (cf. Theorem 4.1) play a central role in 
our proofs. 

Our analysis restricts attention to a linear holding cost structure. How- 
ever, we expect the analysis to go through for some other cost structures 
including convex holding costs. Indeed, the analysis of the convex holding 
cost case will probably be simpler as one tends to get "interior" solutions in 
that case as opposed to the corner solutions in the linear cost case, which 
causes nonsmoothness. One could also enrich the model by allowing aban- 
donment. We expect the analysis to go through with no major changes in 
these cases as well; see the discussion of possible extensions in Section 7. 
For purposes of clarity, however, we chose not to incorporate these addi- 
tional/alternative features because we feel that the current set-up enables 
us to focus on and clearly communicate the main idea: the use of a novel 
Brownian model with state/control dependent diffusion coefficient to obtain 
improved optimality gaps. 

Organization of the paper. Section 2 formulates the model and states the 
main result. Section 3 introduces a (sequence of) Brownian control prob- 
lem(s), which are then analyzed in Section 4. A performance analysis of our 
proposed policy appears in Section 5. The major building blocks of the proof 
are combined to establish the main result in Section 6 and some concluding 
remarks appear in Section 7. 

2. Problem formulation. We consider a queueing system with a single 
server-pool consisting of n identical servers (indexed from 1 to n) and a set 
I = {1, . . . ,1} of job classes as depicted in Figure 1. Jobs of class- i arrive 
according to a Poisson process with rate Aj and wait in their designated 
queue until their service begins. Once admitted to service, the service time 
of a class-z job is distributed as an exponential random variable with rate 
/ij > 0. All service and inter arrival times are mutually independent. 

Heavy-traffic scaling. We consider a sequence of systems indexed by the 
number of servers n. The superscript n will be attached to various processes 
and parameters to make the dependence on n explicit. (It will be omitted 
from parameters and other quantities that do not change with n.) We assume 
that Xf = aiX^ for all n, where A" is the total arrival rate and aj > for 
i £l with ^j Oj = 1. This assumption is made for simplicity of notation and 
presentation. Nothing changes in our results if one assumes, instead, that 
Xf/n — )• Aj and ^Jn{Wl jn — Aj) — )• Aj as n — )• oo where Aj/ X^y^gj A^ = Oj > 0. 
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Fig. 1. A multiclass queue with many servers. 



The nominal load in the nth system is then given by 
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so that defining /i = [^iai/ni]~^ we have that i?" = A"//z, which corre- 
sponds to the nominal number of servers required to handle all the incoming 
jobs. The heavy-traffic regime is then imposed by requiring that the number 
of servers deviates from the nominal load by a term that is a square root 
of the nominal load. Formally, we impose this by assuming that A" is such 
that 



(2) n = i?" + 13V R"" 

for some /3 G (—00, 00) that does not scale with n. Also, we define the relative 
load imposed on the system by class-i jobs, denoted by fj, as follows: 



(3) 



Y.kei'^k/l-i'k 



Note that '^^izx^i — Ij ^'^d Viii can be interpreted as a first-order (fluid) 
estimate for the number of servers busy serving class-i customers. 

2.1. System dynamics. Let Q^{t) and X"(i) denote the number of class-i 
jobs in the queue and in the system, respectively, at time t in the nth system. 
Similarly, let Zf{t) denote the number of servers working on class-i jobs at 
time t. Clearly, for all i,n,t, the following holds: 

xr(t)=zr(t)+Qrw- 
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In our setting, a control corresponds to determining how many of the 
class-i jobs currently in the system are placed in queue and in service for 
i S X. We take the process Z" as our control in the nth system. Note that 
one can equivalently take the queue length process Q" as the control. (The 
knowledge of either process is sufficient to pin down the evolution of the sys- 
tem given the arrival, service processes and the initial conditions.) Clearly, 
the control process must satisfy certain requirements for admissibility, in- 
cluding the usual nonanticipativity requirement. We defer a precise mathe- 
matical definition of admissible controls for now (see Definition 2.2). How- 
ever, it should be clear that, given the process Z", one can construct the 
other processes of interest. 

To be specific, consider a complete probability space (0,7^, P) and 2/ 
mutually independent unit-rate Poisson processes {J\f^{-),Mf{-),i&I) on 
that space. Given the primitives {Mfi-),M,''{-),X^{0),zf{0);i£l) and the 
control process Z^, we construct the processes X"',Q'^ as follows: for t > 
and i G X 

(4) Xrit) = XnO) +K^{X^t)-Mf(^^.J^Zr{s)ds^, 

(5) Qnt)=xr{t)-zr{t). 

The processes Z^,Q",X^ must jointly satisfy the constraints 
(6) (Q"(i), X"(t), Z"(t)) G Z^/, e • Z«(t) < n, 

where e is the /-dimensional vector of ones. 

Controls can be preemptive or nonpreemptive. Under a nonpreemptive 
control, a job that is assigned to a server keeps the server busy until its 
service is completed. In particular, given a nonpreemptive control Z^, the 
process Z^ can decrease only through service completions of class-i jobs. In 
contrast, the class of preemptive controls is broader. While it includes non- 
preemptive policies, it also includes controls that (occasionally) may pre- 
empt a job's service. The preempted job is put back in the queue and its 
service is resumed at a later time (possibly by a different server). Hence, 
the class of preemptive controls subsumes the class of nonpreemptive ones 
(which is also immediate from Definition 1 in [2]) and the cost of an opti- 
mal policy among preemptive ones gives a lower bound for that among the 
nonpreemptive ones. 

In what follows, we will largely focus on preemptive controls, which are 
easier to work with, and derive a specific policy which is near optimal in 
that class. The specific policy we derive is, however, nonpreemptive, and 
therefore, is near optimal among the nonpreemptive policies as well. More 
specifically, the policy we propose belongs to a class which we refer to as 
tracking policies. 
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To facilitate the definition of tracking policies, define lA C M;^ as 
(7) ZY={nG]R^+:^' 



Ui 



Also, for all i and t > 0, let 

(8) X^{t)=X^{t)-y,n. 

Hence, the process X" captures the oscillations of the process X" around 
its "fluid" approximation f^n. Throughout our analysis, for x G M we let 
(x)"^ =max{0,x} and [x)~ =max{0,— x}. 

Definition 2.1. Given a function /iiM — t-ZY, an /i-tracking policy makes 
resource allocation decisions in the nth system as follows: 

(i) It is nonpreemptive. That is, once a server starts working on a job, 
it continues without interruption until that job's service is completed. 

(ii) It is work conserving. That is, the number of busy servers satisfies 
e • Z'^{t) = (e • X"(i)) A n for all t > 0. In particular, no server is idle as long 
as there are n or more jobs in the system. 

(iii) When a class-i job arrives to the system it joins the queue of class i 
if all servers are busy processing other jobs. Otherwise, the lowest-indexed 
idle server starts working on that job. 

(iv) A server that finishes processing a job at a time t, idles if all queues 
are empty. Otherwise, she starts working on a job of class i € IC{t—) with 
probability X^ /Ylik<^K(t-) ^lf> where, for t > 0, the set /C(t— ) is defined by 

(9) /C(t-) = {k eI:Qk{t) - h,{X^{t-)){e ■ X"(i-))+ > 0}. 

Finally, if (e • X"(t— ))+ > and /C(t— ) = 0, she picks for service a customer 
from the lowest index nonempty queue. 

Remark 2.1. For our optimality-gap bounds and, in particular, for the 
proof of Theorem 5.1 it is important that the policy be such that each of the 
job classes in the set IC{t) gets a sufficient share of the capacity. This prevents 
excessive oscillation of the queues that may compromise the optimality gaps. 
Such oscillations could arise if, for example, the policy chooses for service 
a job of class 

i = minargmax{Qfc(i-) - /ife(X''(t-))(e • X"(t-))+ : Qfc(t-) > 0}. 

k£l 

Randomization is just one way to overcome such oscillations and, as the 
proofs (specifically that of Theorem 5.1) reveal, any choice rule that guar- 
antees a sufficient share of the capacity to a class in JC{t—) will suffice. 

Our main result shows that a (nonpreemptive) tracking policy can achieve 
a near optimal performance among preemptive policies. Note that in our set- 
ting under preemption, one can restrict attention to work-conserving poll- 
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cies, that is, policies under which the servers never idle as long as there are 
jobs to work on.^ More precisely, a control is work conserving if the following 
holds for all t > 0: 

(10) e-Q^{t) = {e-X^{t))+. 

Hereafter, we focus on work-conserving controls. Each such control can 
be mapped into a ratio control, which specifies what fraction of the total 
number of jobs in queue belongs to each class. To that end, let 

Qfit) 



(11) unt) 

Note that 
as follows: 



(e-Q"(t))Vl' 
Note that the original control Z" can be recovered from the ratio control [/" 



zr(t)=xr(t)-t/r(t)(e-x"(t))+. 

Equations (4)-(6) can then be replaced by 

(13) Qnt) = Unt){e-X''{t))^, 

(14) Znt) = Xnt)-Q7{t), 

(15) xnt)=xnt)-u,n, 

(16) U'^{t)eU, Q'''{t)£Zl, X"(t)GZ:^. 
Define the filtration 

Tt = <r{Ar,^{s),Aff{sy,iel,s<t} 
and the cr-field 

(17) Too = \/Tt. 

Informally, ^oo contains the information about the entire evolution of the 
processes {Af°',J\ff,i GX). A natural notion of admissibility requires that 
the control is nonanticipative so that it only uses historical information 
about the process X" and about the arrivals and service completions up to 
the decision epoch. To accommodate randomized policies (as the /i-tracking 
policy) we allow the control to use other information too as long as this 
information is independent of ^oo- 



^By a coupling argument, this can be shown to hold with general queueing costs pro- 
vided that there are no abandonments and that the service times are exponential; see, for 
example, the coupling argument on page 1126 of [2]. 



OPTIMALITY GAPS IN THE HALFIN-WHITT REGIME 9 

Definition 2.2. A process U = {Ui{t),t > 0,i G X) is a ratio control for 
the nth system if there exists a process X" = (X"^ , Q'^ , Z^ , X^) such that, 
together with the primitives, (X", U) satisfies (12)-(16). The process U is an 
admissible ratio control if, in addition, it is adapted to the filtration Q V J^" 
where 

r; = a!^K^{X'^s),xr{s),f^. 1^ znu)du, 



MfUH f Zl\u)du\-iel,Q<s<tY 



and ^ is a c-field that is independent of ^oo- The process X" is then said 
to be the queueing process associated with the ratio control U . We let 11" 
be the set of admissible ratio controls for the nth system. 

Ratio controls are work conserving by definition, but they need not be 
nonpreemptive in general. However, note that given a function h:M. ^U, 
the (nonpreemptive) /i-tracking policy corresponds to a ratio control Uh, 
which is nonpreemptive. To be specific, given the primitives and the h- 
tracking policy, one can construct the corresponding queueing process X" = 
{X^,Q^,Z^,X^) (see the construction after Lemma A.l). Then the ratio 
control Uh is constructed using the relation (11) so that X" and Uh jointly 
satisfy (12)-(16). Hence, one can speak of the ratio control and the queueing 
process associated with an /i-tracking policy. Note that since the tracking 
policy makes resource allocation decisions using only information on the 
state of the system at the decision epoch (together with a randomization 
that is independent of the history), the resulting ratio control is admissible 
in the sense of Definition 2.2. The terms ratio control and /i-tracking policy 
appear in several places in the paper. It will be clear from the context 
whether we refer to an arbitrary ratio control or to one associated with an 
/i-tracking policy. 

We close this section by stating the main result of the paper. To that end, 
let 

(18) A'" = {(x,g)GZ^^g = u(e-x-n)+ lor some u £ U} . 

That is, X" is the set on which (X'^,Q^) can obtain values under work 
conservation. In this set e-q = {e-x — n)~^ so that positive queue and idleness 
do not co-exist. We let E^g[-] denote the expectation with respect to the 
initial condition (X^ (0) , Q"^ (0)) = {x,q) and an admissible ratio control U. 
Given a ratio control U and initial conditions {x,q), the expected infinite 
horizon discounted cost in the nth system is given by 



(19) C"(x,5,C/)=E 



u 

x,q 



/•oo 

/ e-T"c-Q"(s)(is 
Jo 
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where c = (ci, . . . , c/)' is the strictly positive vector of holding cost rates and 
7 > is the discount rate. For (x, q) G Af", the value function is given by 



V^{x,q)=mf^Elg 



We next state our main result 



l-OO 



Theorem 2.1. Fix a sequence {{x'^,q^),n S Z+} such that (x",^") e Af" 
and Ix" — vn\ < Ai^/n for all n and some M > 0. Then, there exists a se- 
quence of tracking functions {h"',n G Z+} together with constants C,k>0 
(that do not depend on n) such that 

C'ix'', g", UJi) < Vix"", g") + Clog^ n for all n, 

where UJ^ is the ratio control associated with the h" -tracking policy. 

The constant k in our bound may depend on all system and cost param- 
eters but not on n. In particular, it may depend on (iJ,i,Ci,ai;i G X) and /3. 
Its value is explicitly defined after the statement of Theorem 4.1. 

Theorem 2.1 implies, in particular, that the optimal performance for non- 
preemptive policies is close to that among the larger family of preemptive 
policies. Indeed, we identify a nonpreemptive policy (a tracking policy) in 
the queueing model whose cost performance is close to the optimal value of 
the preemptive control problem. 

The rest of the paper is devoted to the proof of Theorem 2.1, which 
proceeds by studying a sequence of auxiliary Brownian control problems. 
The next subsection offers a heuristic derivation and a justification for the 
relevance of the sequence of Brownian control problems to be considered in 
later sections. 

2.2. Toward a Brownian control problem. We proceed by deriving a se- 
quence of approximating Brownian control problems heuristically, which will 
be instrumental in deriving a near-optimal policy for our original control 
problem. It is important to note that we derive an approximating Brownian 
control problem for each n as opposed to deriving a single approximating 
problem (for the entire sequence of problems) . This distinction is crucial for 
achieving an improved optimality gap for n large because it allows us to 
tailor the approximation to each element of the sequence of systems. 

To this end, let 

/" = A" - fiiUiU for i G I. 

Fixing an admissible control C/" for the nth system [and centering as in (8)], 
we can then write (12) as 

(20) Xr{t)=XriO)+l7t-f,i f\xris) -Uns){e- X-{s))+)ds + Writ), 
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where 

W^t) =Mt{Kt) - \-t + ^,, (\xi\s) - U:{s){e ■ X«(s))+) ds 

In words, W^{t) captures the deviations of the Poisson processes from their 
means. It is natural to expect that an approximation result of the following 
form will hold: {Xf,W^\i G X) can be approximated by {Xf,W^;i G X) 
where 

X^{t) = Xr(0) + l^t - ^i, (\x:{s) - U:{s){e ■ X"(.))+) ds + W^{t), 

Jo 

ft 



W,it) = BtiKt) + B^ (^/x, j^ {X^{s) + v,n - Uns)ie ■ X^s)r)ds 

and B"^, B^ are /-dimensional independent standard Brownian motions. More- 
over, by a time-change argument we can write (see, e.g.. Theorem 4.6 in [9]) 

X^it)=Xr{0) + l?t-f,j' Xns)-Uns)ie-X''{s))+ds 

(22) ^ ^° 

+ 1 ^X'^ + ii^{Xf{s) + v,n - Uns){e ■ l"(s))+) dB,{s), 

where B is an /-dimensional standard Brownian motion constructed by set- 
ting 

^^(^^ ^ /■* dBfifi, f^jX^u) + u,n - U-{u){e • X"(u))+) du) 

■^0 ^/x,(Xf (s) + UiH - U^{s){e ■ X"(s))+) 

, BfjXft) 

Taking a leap of faith and arguing heuristically, we next consider a Brow- 
nian control problem with the system dynamics 

(23) l"(i)=x+ I b''{X''{s),U"is))ds+ [ a'^{X''{s),U"{s))dB{t), 

Jo Jo 

where U"^ will be an admissible control for the Brownian system and 

(24) bnx,u) = l'l-^li{x,-Ui{e■x)+) 
and 



(25) cri{x,u) = ^V^ + ^iUin + iJ,i{xi-Ui{e- x) + ). 
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Note that the Brownian control problem will only be used to propose a can- 
didate policy, whose near optimality will be verified from first principles 
without relying on the heuristic derivations of this section. 

To repeat, the preceding definition is purely formal and provided only 
as a means of motivating our approach. In what follows, we will directly 
state and analyze an auxiliary Brownian control problem motivated by the 
above heuristic derivation. The analysis of the auxiliary Brownian control 
problem lends itself to constructing near optimal policies for our original 
control problem. To be more specific, the system dynamics equation (23), 
and in particular, the fact that its variance is state and control dependent, 
is crucial to our results. Indeed, it is this feature of the auxiliary Brownian 
control problems that yields an improved optimality gap. 

Needless to say, one needs to take care in interpreting (23)-(25), which 
are meaningful only up to a suitably defined hitting time. In particular, to 
have cr" well defined, we restrict attention to the process while it is within 
some bounded domain. Actually, it suffices for our purposes to fix k > and 
m > 3 and consider the Brownian control problem only up to the hitting 
time of a ball of the form 

(26) B2 = {xeR^■.\x\<K^/^log'^n}, 

where | • | denotes the Euclidian norm. We will fix the constant m throughout 
and suppress the dependence on m from the notation. Setting 

(27) n{K) = inf{n G Z+ : a"(x, u) > 1 for all x G i3^, n G ^/}, 

the diffusion coefficient is strictly positive for all n > ?i(k;) and x G i3". Note 
that, for ah i G X, X G 6" and u^U, 

(o-"(x,n))^ > A" + /iii/in-2^i«:A/^log™n, 

so that {af{x,u))'^ > /ijfjn/2 > 1 for all sufficiently large n and, conse- 
quently, ti^k) < oo. 

Remark 2.2. In what follows, and, in particular, through the proof of 
Theorem 2.1, the reader should note that while choosing the size of the ball 
to be en (with e small enough) would suffice for the nondegeneracy of the 
diffusion coefficient, that choice would be too large for our optimality gap 
proofs. 

3. An approximating diffusion control problem (ADCP). Motivated by 
the discussion in the preceding section, we define admissible systems as 
follows. 

Definition 3.1 (Admissible systems). Fix k > 0, n G Z+ and x G M^. 
We refer to 6 = {Q,T, (J-j), P, U, B) as an admissible (k, n)-system if: 

(a) (fi, J^, (J^t),P) is a complete filtered probability space. 
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(b) B{-) is an /-dimensional standard Brownian motion adapted to (J-t)- 

(c) U is U-valued, J^-measurable and (J-t) progressively measurable. 

The process U is said to be the control associated with 9. We also say 
that X is a controlled process associated with the initial data x and an 
admissible system ^ if X is a continuous (J"i)-adapted process on {Q,T,F) 
such that, almost surely, for i < f", 

X{t) = x+ I b'^{X{s),U{s))ds+ I a''{X{s),U{s))dB{t), 
Jo Jo 

where b"'^-,-) and a"'{-,-) are as defined in (24) and (25), respectively, and 

f^ = inf{i > : X{t) ^ B]^}. Given k > and n e Z+, we let @{k, n) be the 

set of admissible (K,n)-systems. 

The Brownian control problem then corresponds to optimally choosing an 
admissible (K,n)-system with associated control {U{t),t>0) that achieves 
the minimal cost in the optimization problem 



(28) y"(x,K)= inf El 



r e-"f^y^CiUi{s){e-X{s))+ds , 

where IE^[-] is the expectation operator when the initial state is x G M^ and 
the admissible system 9. Hereafter, we refer to (28) as the ADCP on B". 
The following lemma shows that the Definition 3.1 is not vacuous. The proof 
appears in the Appendix. 

Lemma 3.1. Fix the initial state x G M , k > 0, n> n{K) and an ad- 
missible {K,n) -system 9. Then, there exists a unique controlled process X 
associated with x and 9. 

To facilitate future analysis, note from the definition of f^ and (28) that 

(29) 1/"(x,K)<i(e-c)KV^log™n. 

7 



Definition 3.2 (Markov controls). We say that an admissible (K,n)- 
system 9 = (0, J^, (J"t),P, [/, i?) with the associated controlled process X" 
induces a Markov control if there exists a function g^{-) :B^ -^lA such that 
i/(t) = c,"(X"(t)) for t<fJ^.We extend the function 5" to M^ as follows: 

g^ix), x^Bl, 



' ' ei, otherwise 

where e\ is the /-dimensional vector whose first component is 1 while the 
others are 0. We refer to /i"'(-) as the tracking function associated with the 
admissible system 9. 
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In what follows, a policy U will be called optimal for the approximating 
diffusion control problem (ADCP) on S" if there exists an admissible {K,n)- 
system 6 = {n,T, (-Ft),P, U,B) such that 



F"(x,K)=Ef 



■^0 i&x 



Recall that X and U are used to denote performance relevant stochastic 
processes in both the Brownian model and the original queueing model, and 
that we add a hat, that is, we use X and U in the context of the Brownian 
model. To avoid confusion, the reader should keep in mind that hat-processes 
correspond to the ADCP while the ones with no hats correspond to the 
original queueing model. 

Roadmap for the remainder of the paper. The main result in Theorem 2.1 
builds on the following steps: 

1. In Section 4 we show that for each n, the HJB equation associated with 
the ADCP has a unique and sufficiently smooth solution. Using that so- 
lution we advance an optimal Markov control for the ADCP together 
with the corresponding tracking function. We also identify useful gradi- 
ent bounds on the solutions to the sequence of HJB equations; cf. Theo- 
rem 4.1. 

2. In Section 5 we conduct a performance analysis of /i-tracking policies in 
the queueing system; cf. Theorem 5.1. 

3. The result of Theorem 5.1 together with the gradient estimates in Theo- 
rem 4.1 are combined in a Taylor expansion- type argument in Section 6 
to complete the proof of Theorem 2.1. 

As a convention, throughout the paper we use the capital letter C to 
denote a constant that does not depend on n. The value of C may change 
from line to line within the proofs but it will be clear from the context. 

4. Solution to the ADCP. This section provides a solution for the ADCP 
on i3" for each n G Z and k > 0. The HJB equation is a fully nonlinear and 
nonsmooth PDE. As such, it requires extra care when compared with the 
usual semilinear PDEs that arise in the analysis of asymptotically optimal 
controls in the Halfin-Whitt regime. We will build on existing results in 
the theory of PDEs and proceed through the following steps: (a) establish 
the existence and uniqueness of classical solutions; (b) relate this unique 
solution to the value function of the ADCP and (c) establish useful gradient 
estimates on the solution for the HJB equation. The last step is not necessary 
for existence and uniqueness but is important for the analysis of optimality 
gaps. 
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In what follows, we fix k > and n > n{K) and suppress the dependence 
of the solution to the HJB equation on n and k. The following notation is 
needed to introduce the HJB equation. Given a twice continuously differen- 
tiable function 0, define 

(Pi = -K— and (pa = —-^. 
dxi dxf 

Also, define the operator A" for uGU as follows: 

(31) A-^cP = Y,b7i;u)cP, + ^Y.^an;u)f^u. 

Defining 

L{x,u) = y CiUi{e-x)~^ 

for X € M^ and u ^U, the HJB equation is given by 

(32) 0= inf {L(x,n) + A:J0(x) -7,/.(x)}. 

Substituting 6"(-,-) and o-'"(-,-) into (32) gives 

= -70(x) + (e • x)+ • min<^ Cj + /Uj0j(x) - -fJ,ict)ii{x) } 
(33) 

Our analysis of the HJB equation (33) draws on existing results on fully 
nonlinear PDEs, and, in particular, the results on Bellman-Pucci type equa- 
tions; cf. Chapter 17 of [6]. 

In what follows, fixing a set S C R;^, C'^{B) denotes the space of twice 
continuously differentiable functions from B to M. For u G C^(i3), we let Du 
and D^u denote the gradient and the Hessian of u, respectively. The space 
C^'"(i3) is then the subspace of €"^{3) members of which also have second 
derivatives that are Holder continuous of order a. That is, a twice continu- 
ously differentiable function ii:M^ — )■ M is in C'^'°'{B) if 

sup j^ ^^^' < oo, 

where | • | denotes the Euclidian norm. We define d^ = dist(2;, dB) = inf{|x — y\ 
y G dB} where dB stands for the boundary of ,6 and we let dx^z = ^ii-{dx,dz}- 
Also, we define 



+ > XC - tJ-iXi)(j)i{x) + - ^(A" + mii^iu + Xi))(l)ii{x). 



2 



(34) \u\la,B = J2^^^ls+ sup d 



,^,jDMx)-DMy)\ 



j=0 x,yeB,x^y ' F y\' 
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where [u]* ^ = sup^g^ di:\D^u{x)\ for j = 0, 1, 2. Note that di denote the jth. 
power of dx and, similarly, d^"^" is the (2 + a)th power of dx^y Finally, we 
let \u\Iq = [u]*q^i3 = suPx^i3Hx)\. 

In the statement of the following theorem, ej is the /-dimensional vector 
with 1 in the jth place and zeros elsewhere. Also, B^, m and n{K) are as 
defined in (26) and (27), respectively. 

Theorem 4.1. Fix k> and n > n(K). Then, there exists < a < 1 
(that does not depend on n) and a unique classical solution (/>" ^C^'^{B]^) n 
C^'"(S") to the HJB equation (33) on B^ with the boundary condition 0" = 
on 96". Furthermore, there exists a constant C > (that does not depend 
on n) such that 

(35) \rMo.,B-^<CV^\og'''^n, 

where k^ = 4m(l + 1/a). In turn, for any -d <1, 

with ki = ko — m and k2 = kg — 2m. Also, 



(36) snp \D^:{x)\<-^log'^^n and sup \D^r.{x)\ < -^^^-^^ 



xeej^ J- - (^ xGBg„ 



sup 

ueu 






(37) ^ 

for all x,y £ B^^ with \x — y\ < 1. 

Note that (36) follows immediately from (35) through the definition of the 
operation | • I2 « b" ™ (^4). Henceforth, we will use ki,i = 0,l,2 for the values 
given in the statement of Theorem 4.1. Moreover, the constant k appearing 
in the statement of Theorem 2.1 is equal to ko + 3. 

Theorem 4.1 facilitates a verification result, which we state next followed 
by the proof of Theorem 4.1. Below, V"'{x,k) is the value function of the 
ADCP; cf. equation (28). 

Theorem 4.2. Fix k> and n > u^k). Let 0" be the unique solution to 
the HJB equation (33) on B^ with the boundary condition (/>" = on dB^ . 
Then, (/>]J(x) = V'^{x,k) for all x G S". Moreover, there exists a Markov 
control which is optimal for the ADCP on i3". The tracking function h%'^ 
associated with this optimal Markov control is defined by /ik'^(x) = Cju/^,), 
where 

(38) i"-{x) = minargmin<^ ( a + iJ.i{(f)'^)i{x) - -iJi{((>]^)ii{x) j (e • x)" 
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The HJB equation (33) has two sources of nondifferentiabihty. The first 
source is the minimum operation and the second is the nondifferentiabihty 
of the term (e • x)^ . The first source of nondifferentiabihty is covered almost 
entirely by the results in [6]. To deal with the nondifferentiabihty of the 
function (e • x)+, we use a construction by approximations. The proof of 
existence and uniqueness in Theorem 4.1 follows an approximation scheme 
where one replaces the nonsmooth function (e • x)"*" by a smooth (param- 
eterized by a) function fa{e-x). We show that the resulting "perturbed" 
PDE has a unique classical solution and that as a — t- oo the corresponding 
sequence of solutions converges, in an appropriate sense, to a solution to (33) 
which will be shown to be unique. Note that this argument is repeated for 
each fixed n and k. 

To that end, given a > 0, define 



(39) faiy) 



2 1 1 1 1 

1 loa 4o 4a 



, 0, otherwise. 

Replacing (e • x)"*" with /^(e • x) in (33) gives the following equation: 

= -7(/)(x) + /a(e • x) • min<^ Ci + iJ.i4>i{x) - -fii<t)ii{x) } 

iex I z I 

(40) \ 

+ X](^«" ~ HiXi)(pi{x) + - ^(A" + fiiiuiu + Xi))(/)ii{x). 

To simphfy this further, let T = fi^! x M+ x M^ x M^^^ and for ah y G F, 
define the function 

(41) F^ly]=min{Fi[y],...,Fi[y]}, 
where for k £Z and y = (x, z,p, r) £ T, 

F^{y] = fa{e-x 



1 
Ck + IJ'kPk - -^fJ-krkk 



(42) 






+ 2 '^(K + /"j(i^j"' + Xi))rii 
iex 

Then, (40) can be rewritten as 



(43) Fa[x,u{x),Du{x),D''u{x)] = 0. 

In the following statement we use the gradient notation introduced at the 
beginning of this section. 
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Proposition 4.1. Fix k > 0, n> n{K) and a> Q. A unique classical 
solution 0;!^„ gC°'^(^;!) nC2'"(e;}) exists for the PDE (40) on B'^ with the 
boundary condition 0" ^ = on 96" . Moreover, 

(44) IC,al2,a,B." < C|C,alS,B2 ^Og'" ^ ^ ^ 

for ko = Am{l + 1/a) where < a < 1 and C > do not depend on a and 
n and C does not depend on a. Also, (ff^^ is Lipschitz continuous on the 
closure i3" with a Lipschitz constant that does not depend on a (but can 
depend on k and n). 

We postpone the proof of Proposition 4.1 to the Appendix and use it to 
complete the proof of Theorem 4.1, fohowed by the proof of Theorem 4.2. 

Proof of Theorem 4-1- Since we fix n and k, they will be suppressed 
below. We proceed to show the existence by an approximation argument. 
To that end, fix a sequence {o'^; /c E Z} with a''" — >• oo as A; — >• oo and let (p^k 
be the unique solution to (40) as given by Proposition 4.1. The next step is 
to show that (j)^k has a subsequence that converges in an appropriate sense 
to a function (j), which is, in fact, a solution to the HJB equation (33). To 
that end, let 

(45) C2'"(6) = {ue C^'^'iB) : |u|^,,,g < oo}. 

Then, C*'"(;B) is a Banach space (see, e.g.. Exercise 5.2 in [6]). Since the 
bound in (44) is independent of a, we have that {4>ai^} is a bounded sequence 
in C^,'"(i3) and hence, contains a convergent subsequence. Let u be a limit 
point of the sequence {^qS:}. Since the gradient estimates in Proposition 4.1 
are independent of a, they hold also for the limit function u, that is, 

(46) \u\l^^i,<C\u\yiog'°n<C 

for constants a and C that are independent of n. Proposition 4.1 also guar- 
antees that the global Lipschitz constant is independent of a so that we may 
conclude that u € C^'^{B) and that u = on dB. 

We will now show that u solves (33) uniquely. To show that u solves (33), 
we need to show that F[u\ =0 (where F[-] is defined similar to Fa[-] with 
(e • x)~^ replacing fa{e ■ x)). To that end, let {a'^, /c G Z} be the corresponding 
convergent subsequence [i.e., such that cp^k — )• u in C*'"(6)]. Henceforth, to 
simplify notation, we write 

Kl'l i^Pa^l i^)] = ^ah [X, (kah {x).D<j)^ki {x),D'^(p^k, (x)] 

(and similarly for F[-]). Fix5>0 and let B{6) = {xeM.^ : |2;| < Ky/n log^ n — S} . 
Note that since (p^ki — >• u in C^'°'{B) we have, in particular, the conver- 
gence of {4>^ki{x),D(l)^ki{x),D'^(j)^ki{x)) ^f {u{x),Du{x),D'^u{x)) uniformly 
in X £ B{6). The equicontinuity of the function F'^[-] on T guarantees then 



OPTIMALITY GAPS IN THE HALFIN-WHITT REGIME 19 

that 

(47) \F,H{<t>a^A^)]-Fa'^A<m<e 

for all I large enough and x G B{5). Note that sup^.gig/ |/a(e " 3^) — (e • x)~^\ < e 
for all a large enough so that, 

(48) sup|F„.Jn(x)]-FKx)]|<e 

for all / large enough. Combining (47) and (48), we then have 
sup|F^.,K.,(x)]-FKx)]|<2e 

xGB 

k 

for all / large enough and x G B{6). By definition F° [</>„*;; (x)] = for all 
X € B and since e was arbitrary we have that F[u{x)] = for all x G B{6). 
Finally, since 6 was arbitrary we have that F[u{x)] = for all x £ B. We 
already argued that n = on dB, so that u solves (33) on B with u = 
on dB. This concludes the proof of existence of a solution to (33) that 
satisfies the gradient estimates (35). 

Finally, the uniqueness of the solution to (33) follows from Corollary 17.2 
in [6] noting that the function F[x,z,p,r] is indeed continuously differen- 
tiable in the {z,p,r) arguments and it is decreasing in z for all {x,p,r). 

Using Theorem 4.2 [which only uses the existence and uniqueness of the 
solution (j)]l{x) that we already established] together with (29) we have that 

ldo,B" = sup V"'{x,k) < -K^/nlog'^n. 

xeBu 7 

The bounds (35) and (36) now follow from (46) and we turn to prove (37). 
To that end, since (^" solves (33), fixing x,y £B^ we have 



(49) 



2 Z](^^ + -"^^^^^ + ^0)(C)n(a^) - 2 IZ(^^ + ^^(^^" + y0)(O«(y) 
<7lC(^)-C(y)l 
+ 



(e 


■x^. 


min< Ci 

i&x y 


+ /.,(C).(^)- 


^M^(C)n(^)} 


- 


(e-y) 


^ •min< 

i&X 


Ci + ^i,{(t)lUy) 


-\^^^{'PlUy)] 



We will now bound each of the elements on the right-hand side. To that end, 
let i{x) be as defined in (38) and for each x,z £ B^^ define 

^^i(x)(^) =Ci(x) +/^j(x)(<^")i{x)(^) - 2H^)i^K)i{x)i{x){z)- 

Using (36), we have by the mean value theorem that 

(50) |C(x)-C(?/)l<k-y|max sup \{rM^)\ < C log"- n 
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for all x,y £ B^^ with |x — y| < 1, and we turn to bound the second element 
on the right-hand side of (49). Here, there are two cases to consider. Suppose 
first that i{x) = i{y) = i. Then, using (36) and the mean value theorem we 
have 

UrMx) - (C)i(y)l < k - y| max sup \ir.Uz)\ < c-°^ ' "^ 



and, in turn, that 

(51) |Mf(x)-M;^(y)|<C7V''' 



n 



for all x,y £ B"^^ with |x — y| < 1. Now, \x\ V|y| < K-y/nlog^n for all x,y £ B^^ 
and, by (36), sup.gg^^ \{<m^^{z)\ V |(C)*(^)I < Clog'^^ n so that 

|(e.x)+Mr(x)-(e.y)+Mf(y)| 

(52) <K./7i\og'^n\M't{x)-M't{y)\+ sup |Mf(z)| 

<Clog'=in. 
If, on the other hand, i{x) ^ i{y) then by the definition of i{-), 

and 

Ci(y) + fJ-i{y)iK)i{y)iy) " 2 ^^i{y) (^k) i{y)i{y) iv) 

<Ci^x) +/^J(a;)(Oj(x)(y) - 2/^*(a;)(Oj(a:)i{x)(?/)- 

That is, 

(53) M;^,)(x)<M;^^)(x) and MJl^^{y) < MJl^^{y) . 

Using (36) as before we have for x,y £ B^^ with \x — y\ <1 and i{x) / i{y) 
that 

iM^a^) - Ml^{y)\ + |M;|'^)(rr) - M^^iy)] < C-^"^'' ^ 



/n 
By (53) we then have that 

\Mllx)i^) - M^^y^{y)\ < |M;^,.)(x) - M;^,.)(y)| 

+ |M;^^)(x)-M;^^)(y)| 
^ ^log'^^ „ 



n 
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for all such x and y. In turn, since |x| V \y\ < K^/nlog"^ n, 
(54) |(e • x)+M;^,)(x) - (e • y)+M;^^)(y)| < Clog^^ n 

for 3;,y G 5^^ with \x — y\ <1 and i(a;) 7^ i(y). Plugging (50), (52) and (54) 
into the right-hand side of (49) we get 

- ^(A^ + ^^^{uin + Xi))(On(a^) - 2 I^(^r + /"i(^i^ + yi))(0")n(y) 

<Clog'=in 
for all a;,y G i3,^^ with |x — y| < 1. Finally, recall that 



ai{x,u) = ^ Xl + ^iUiU + ^ii{xi -Ui{e-x)+) 
so that for all u £l/(, 

- {(l)'^)ii{x){X^ + Hii^iTi + iJ,i{xi - Ui{e ■ x)+)) 

- ^(A^ + M,(z^in + x,))(0«(a^) 



< 



iex 



+ 



+ 



-J^(A7+/i,(z.in + yi))(C)n(y) 

2 X]('^«)^^(^)^*^*(^ ■ ^^"^ ~ ('/'")«(y)^i'"i(e • y)' 

iex 
i^^i(l>'^)iiiy)f^iixi - yt) . 



iei 



The last two terms above are bounded by Clog ^ n by (36) and using |3;| V 
\y\ < Ky/nlog"^ n. Together with (55) this establishes (37) and concludes the 
proof of the theorem. 

Proof of Theorem 4-2. Fix an initial condition x E B^^ and an admissible 
(k, n)-system 6 = (fi, J^, (J^j), P, U, B) and let X" be the associated controlled 
process. Using Ito's lemma for the function ip{t,x) = e~^*(f)'^{x) in conjunc- 
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tion with the inequahty 

L{x,u) + Au(l)'^{x) - jcp'^ix) >0 for all x€B2,uGU 
[recall that c^" solves (33)] we have that 

(P'^{x)<El / e-TU(X"(s),;7(s))ds + E^e-^(*^^")</.;!(X"(tAf^)) 



(56) 

Here, f" is as defined in Definition 3.1 and it is a stopping time with respect 
to {Ft) because of the continuity of X". We now claim that 

E^[e-^*^^"</.:(X"(t A f^))] ^0 as t ^ oo. 

Indeed, as (/>" is bounded on B", on the event {f" = oo} we have that 

e-7(tAf.")<^n(X(tAf^))^0 ast^oo. 

On the event {f" < oo} we have X"'(f") G dB and, by the definition of f", 
that 0"(X"'(f")) = 0. The convergence in expectation then follows from the 
bounded convergence theorem (using again the boundedness of <^" on ^"). 
The last term in (56) equals zero by the optional stopping theorem. 

Letting i — )• oo in (56) and applying the monotone convergence theorem, 
we then have 



<t>l{^)<K 



e"'<'L{X''is),U{s))ds 



Since the admissible system 6 was arbitrary, we have that </>"(x) < ^"'(x, k). 
To show that this inequality is actually an equality, let 

(57) /i"(x) = ejn(^), 

where eini^\ is as defined in the statement of the theorem. 

The continuity of cj)^ guarantees that the function i"(x) is Lebesgue mea- 
surable, and so is, in turn, /i"(-). Consider now the autonomous SDE: 

(58) X'^(t) = x+ / 6"(l"(s))ds+ I a''{X''{s))dB{s), 

Jo Jo 

where 5"(y) = 6'"(y, K'^{y)) and cF^{y) = &"'{y, h'^{y)) on i3". Then, 6" and a" 
are bounded and measurable on the bounded domain B"^. Also, as the ma- 
trix o"" is diagonal and the elements on the diagonal are strictly positive 
on i3", it is positive definite there. Hence, a weak solution exists for the 
autonomous SDE (see, e.g.. Theorem 6.1 of [10]). In particular, there exists 
a probability space (ri,^,P), a filtration [Qt) that satisfies the usual condi- 
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tions, a Brownian motion B{t) and a continuous process JC^ — both adapted 
to (Gt), so that X^ satisfies the autonomous SDE (58). Finahy, since X" 
has continuous sample paths and it is adapted, it is also progressively mea- 
surable (see, e.g., Proposition 1.13 in [9]) and, by measurability of /i"(-), so 
is the process U{t) = hl{X''{t)). Consequently, 6 = {n,g,gt,F,U,B) is an 
admissible system in the sense of Definition 3.1 and X" is the corresponding 
controlled process. 

To see that 9 is optimal for the ADCP on i3", note that for s < f", we 
have by the HJB equation (32) that 

L(X"(s), U{s)) + A^^^^r.iX^(s)) - 7'^"(^"(«)) = 0. 

Applying Ito's rule as before, together with the bounded and dominated 
convergence theorems, we then have that 



and the proof is complete. 



e-^U(X"(s),C/(s))ds 



5. The performance analysis of tracking policies. This section shows 
that given an optimal Markov control policy for the ADCP together with 
its associated tracking function /ik", the nonpreemptive tracking policy im- 
itates, in a particular sense, the performance of the Brownian system. 

Theorem 5.1. Fix n and k' < k as well as a sequence {{x'^,q^),n G Z+j 
such that (x",g"') G Af", and jx" — z^n| < My/n for all n and some M > 0. 
Let (p^ and h^^ he as in Theorem 4-2 and define 

r{x^u)=L{x,u)+Al4>l{x)-^4>l{x) forxGB:,u€U. 

Let UJ^ be the ratio control associated with the h^'^ -tracking policy and let 
X" = (X"' , Q"^ , Z"^ , JC^) be the associated queueing process with the initial 
conditions Q"'{0) = q^ and X"'{0) = x"" — un and define 

r,V = inf{t>0:X"(t)^^;:,}Ariogn. 

Then, 



.T 



e-^'\r{x''{s),ui:{s))-r{x''{s),hr{x''{s)))\ds 





E 

for a constant C that does not depend on n 



<C71og^o+3 



n 



Theorem 5.1 is proved in the Appendix. The proof builds on the gradient 
estimates in Theorem 4.1 and on a state-space collapse- type result for certain 
sub-intervals of [0,t", j\. 
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Remark 5.1. Typically one establishes a stronger state-space collapse 
result showing that the actual queue and the desired queue values are close in 
supremum norm. The difficulty with the former approach is that the tracking 
functions here are nonsmooth. While it is plausible that one can smooth 
these functions appropriately (as is done, e.g., in [2]), such smoothing might 
compromise the optimality gap. Fortunately, the weaker integral criterion 
implied by Theorem 5.1 suffices for our purposes. 

6. Proof of the main result. Fix k > and let cf)^ be the solution to (33) 
on B^ (see Theorem 4.1). We start with the following lemma where 6^(-, •) 
and a'^{-^-) are as in (24) and (25), respectively. 

Lemma 6.1. Let f/" be an admissible ratio control and let X" = (X", Q", 
Z",X") be the queueing process associated with U^ . Fix k' < n and T > 
and let 

T^,^r = inf{t > : X"(t) ^ B]^,} A Tlogn. 

Then, there exists a constant C that does not depend on n (but may depend 
on T , K and k' ) such that 



E[e-'^V.-0:!(X"(r:, ^))] < C(^"(0)) +IE 



■7E 



e-^M^„(,)C(^"(^))^« 



-7S ,n. 



<Pl{X-{s))ds 



+ C\og''^+'n 



fci+i, 



< E[e"'^ «'.'rC(^"(^^',T))] +2Clog'=i n- 

We will also use the following lemma where c = (ci, . . . ,ci) are the cost 
coefficients (see Section 2). 

Lemma 6.2. Let {x"',q"') be as in the conditions of Theorem 2.1. Then, 
there exists a constant C that does not depend on n such that 



(59) 



E^ 



-7s 



{e-c){e-X''{s))^ds 



<C\og^n 



and 

(60) E^„,,„[e-^^"'.-,^::(X-(r:, J,))] < Clog^n 

for all n and any admissible ratio control U . 



We postpone the proof of Lemma 6.1 to the end of the section and that 
of Lemma 6.2 to the Appendix and proceed now to prove the main result of 
the paper. 
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Proof of Theorem 2.1. Let /ik*^ be the ratio function associated with the 
optimal Markov control for the ADCP (as in Theorem 4.1). Since k is fixed 
we omit the subscript k and use /i" = h^^ . Let U^ be the ratio associated 
with the /i"-tracking policy. 

The proof will proceed in three main steps. First, building on Theorem 5.1 
we will show that 



(61) E 



e-<^L{X'-{s),Uj:{s))ds 



<C(^"(0)) + Clog 



feo+3 



n. 



Using Lemma 6.2, this implies 

C7"(x",(?",?7^")=E 



(62) 



e-'^'L{X^{s)M{s))ds 



<C(^"(0)) + Clog' 



,A:o+3 



n. 



Finally, we will show that for any ratio control C/"", 



(63) C(^"(0))<IE 



-7s 



L(X"(s),[/"(s))ds 



+ C71og^i+'n 



fci+i, 



where we recall that ki = kQ — m. In turn, 

^"(2;",g") > C(^" - ^n) - Clog^'i+^n > C''{x'',q'\Uj^) - 2Clog'''+^n, 

which establishes the statement of the theorem. 
We now turn to prove each of (61) and (63). 

Proof of (61). To simplify notation we fix k > throughout and let 
/i"(-) = /ik*^. Using Lemma 6.1 we have 

-7t" 



ne-''-''-€{X''ir:,T))] 



(64) <C(^"(0))+E 



■7E 



e-^^^^,"wC(^"(s))d. 



<^l{X-{s))ds 



+ Clog'=i+^n. 



From the definition of /i" as a minimizer in the HJB equation we have that 



= E 



-isjj^n 



lHx-is))rAX-{s))ds 



-7E 
+ E 



<^l{X-{s))ds 



e-^'L{X''{s),hJ'{X''{s)))ds 
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By Theorem 5.1 we then have that 



(65) 





■7E 
■E 



e-^M^„(,)C(^"(^))d« 



e-^V:!(X"(s))ds 






s)) ds 



>0. 



Smce (/)" is nonnegative, combining (64) and (65) we have that 



E 



e-^'L{X^{s),Uj:{.s))ds 



<C(^"(0)) + Clog'=«+3^, 



which concludes the proof of (61). 



Proof of (63). We now show that y"(a;,g) > (/>;?(X"(0)) - Clog^'^+^n. 
To that end, fix an arbitrary ratio control V^ and recall that by the HJB 
equation, 

A>:!(x)-7C(^)+M^,«)>o 

for all ti E ZY and x G S". In turn, using the second inequality in Lemma 6.1 
we have that 

E[e-^^"'.-0:!(X"(T:,^))] 



>C(^"(0))-E 



2C71og'=i+^n. 



e-^U(X"(s),C/"(s))ds 







Using Lemma 6.2, we have, however, that 

E[e-^^«'.-C(^"(r:,,T))]<C^log'n 
for a redefined constant C so that 

r ^t" 



Clog^n>C(^"(0))-IE 
-2Clog'=i+in 

>0:!(x"(o))-E 

-2Clog'=i+^n 







e-^U(X"(s),C/"(s))ds 



e-^^L(X"(s),[/"(s))(is 



and, finally, 

C(^"(0))<E 



e-^^L(X"(s),C/"(s))(is 



+ C71og''i+^n 



fe-i+i , 
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for a redefined constant C > 0. This concludes the proof of (63) and of the 
theorem. 

We end this section with the proof of Lemma 6.1 in which the following 
auxiliary lemma will be of use. 

Lemma 6.3. Fix k > and an admissible ratio control U^ and let X"" = 
{X"',Q"',Z'^,X"') be the corresponding queueing process. Let 

<r = inf{t>0:X"(t)^C}Ariogn, 

and (W^^ji £ I) be as defined in (21). Then, for each i £l, the process 
W^{- At"„) is a square integrable martingale w.r.t to the filtration (-T^at" ) 
as are the processes 

Ar" 






A1-(.) = (W^-(.Ar,»r- / {cfi{X^{s\\J^\s))Yds 

Jo 

and 



Lemma 6.3 follows from basic results on martingales associated with time- 
changes of Poisson processes. The detailed proof appears in the Appendix. 

Proof of Lemma 6.1. Note that, as in (20), X" satisfies 

X^{t) = X^{^)+ fh-{X-{s),U-{s))ds + W^{t), 
Jo 

and is a semi martingale. Applying Ito's formula for semimartingales (see, 
e.g.. Theorem 5.92 in [14]) we have for all t < r", j,, that 

e-^V"(^"(i)) = C(^"(0)) 

+ E e-^^[C(^"(s))-C(^"(«-))] 

s<t: |AX"{s)|>0 



-E E e-^^(0«)r(x"(.))Axr(.) 

i&^ s<t: \AX"{s)\>0 

+Er^'''('^")^(^"(^-))^"(^"(^)'f^"(^))^^ 

.•^T Jo 



J [\-^^r.{X-is))ds 
Jo 
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and, after rearranging terms, that 



i&l- s<t: |AX"(s)|>0 






-7S('^"V('V" 



-MX''{s-)MX-{s),U-{s))ds 



Jo 



where 



C'it) 



E 



s<t: |AX"(s)|>0 



rAX''{s))-ct>l{X-{s-)) 

-j;(C),(x"(.-))Axr(s) 

Setting t = tJ}, rp as defined in the statement of the lemma and taking expec- 
tations on both sides we have 

E[e-^^"'.-<^:!(X"(t))] 

(66) 

-I- 
2 



iei 



-y^(rh'^\-(y 



lUX-{s-m{X^{s),U-{s))ds 



s<t: |AX"(s)|>0 



iex 



+ E[C"(r:,^)]-7E 



' e-^^<l^l{X^\s))ds 



We will now examine each of the elements on the right-hand side of (66). 
First, note that AX"(s) = ATy"(s) and, in particular, 



E 



E 



e-^%r.),,{X-{s-)){AX^{s)f 



s<r" ^:|AX"(.)|>0 



:E 



Y, e-^%r.UX^{s-)){AW^{s)) 



■s<r^, y:|AX"(s)|>0 



OPTIMALITY GAPS IN THE HALFIN-WHITT REGIME 



29 



Using the fact that V", as defined in Lemma 6.3, is a martingale as weh as 
the fact that (/)"(X'"(s)) and its derivative processes are bomided up to r", , 
we have that the processes 

■At" „ 



(67) 
and 



vr(-) := 



-75/ ,n 



^,ux^{s-))dvns) 



■At" 



Jo 

are themselves martingales with V"(0) = 7W"(0) = and in turn, by optional 
stopping, that E[Vf (r^, -p)] = E[>(^(r^, ,r)] (see, e.g., Lemma 5.45 in [14]). 

In turn, by the definition of A^"(-) and V"(-) we have 



E 



Y^ e-^^mu{X-{s-)){AWr{s)) 



's<tJ^, ^■. |AX"(s)|>0 



:E 



:E 



nf„\\2 



{<t>:ux-{s-))d{wr{s)) 



-^strn 



'lUX^{s-)){a^{X^{s),U^\s))Yds 



Plugging this back into (66) we have that 

E[e-'^^"'.TC(^"(i))] 



C(^"(0)) + 5]E 
is: 



ex 

.71 



-T«/'^"V/'i"" 



UX-{s-m{X^{s),U-{s))ds 



-IS (in 



l)u{X-{s-)){a^{X-{slU-{s))Yds 



iex 

7E 



' e-^^l{X^{s))ds 







+ E[C"(r:,^)], 



which, using the definition of A^ in (31), yields 



Efe" 



'"I'^.-i Tj.nrvn 



C(^"(i))] 

b:iX-iO))+E 



e-^M^„(,)C(^"(s))^^ 



7E 



e-T>-(X"(.))ds 



+ E[C"(r:,^)]. 
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To complete the proof it then remains only to show that there exists a con- 
stant C such that 

|E[C"(T:,,^)]|<C71og^^+in. 

To that end, note that by Taylor's expansion, 

ct>i{x-{s)) = r.ix'^is-)) + 5^(c).(^"(^-))Axr(s) 



+ 1 Y.(^:ux^{s-) + r?;f„(,_))Axr(s), 



iex 



where r/jf„(j,_\ is such that X"(s—)-|-7yj^„/_-j is between X"(s—) and X"'{s—) + 
AX"'{s). In turn, adding and subtracting a term, we have that 



(69) -lY.(^:UX-{s-)){AXns)f 

= Y^l{{KUX''{s-) + ^x-^,^^)-{KUX''{s-mAXns)f. 



iei 



Since the jumps are of size 1 and, with probability 1, there are no simulta- 
neous jumps, we have that |??y"(s-)I — 1- Adding the discounting, summing 
and taking expectations we have 



E[C"(t)] 
(70) < E 



E 



-7s 



s<t: |AX"(s)|>0 



Y,l max((C)u(X"(.-) + y) 



b^^UX^is-MAX^s))' 



and a lower bound can be created by minimizing over y instead of maximiz- 
ing. Using again the fact that AXf (t) = AW^{t) and that Mf and Vf as 
defined in (68) and (67) are martingales, we have that 



E[C"(t)]<E 



iei 



(71) 



t^l max((C)n(^"(s-)+y) 
^{a^{X^[s),U^{s))fds 
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From (37) we have that 



(72) \ 



Vr 



^{{<t^l)ii{y)-{<t^lUx)){a^{x,u)y 



fel 



<C71og*^in 



for all u G ZY and x, y G ;B", with \x — y\<l. The proof is then concluded by 
plugging (72) into (71), setting t = t^, rp and recalling that we can repeat 
all the above steps to obtain a lower bound in (71) by replacing niaXy. \y\<i 
with minj^.|y|<i in (70). 

7. Concluding remarks. This paper proposes a novel approach for solv- 
ing problems of dynamic control of queueing systems in the Halfin-Whitt 
many-server heavy-traffic regime. Its main contribution is the use of Brow- 
nian approximations to construct controls that achieve optimality gaps that 
are logarithmic in the system size. This should be contrasted with the opti- 
mality gaps of size o{y/n) that are common in the literature on asymptotic 
optimality. A distinguishing feature of our approach is the use of a sequence 
of Brownian control problems rather than a single (limit) problem. Having 
an entire sequence of approximating problems allows us to perform a more 
refined analysis, resulting in the improved optimality gap. 

In further contrast with the earlier literature, in each of these Brownian 
problems the diffusion coefficient depends on both the system state and the 
control. Incorporating the impact of control on diffusion coefficients allows 
us to track the performance of the policy better but, at the same time, it 
leads to a more complex diffusion control problem in which the associated 
HJB equation is fully nonlinear and nonsmooth. For each Brownian prob- 
lem, we show that the HJB equation has a sufficiently smooth solution that 
coincides with the value function and that admits an optimal Markov pol- 
icy. Most importantly, we derive useful gradient estimates that apply to the 
whole sequence and bound the growth rate of the gradients with the system 
size. These bounds are crucial for controlling the approximation errors when 
analyzing the original queueing system under the proposed tracking control. 

The motivating intuition behind our approximation scheme is that the 
value functions of each queueing system and its corresponding Brownian 
control problem ought to be close. In particular, the optimal control for the 
Brownian problem should perform well for the queueing system. Moreover, 
the optimal Markov control of the Brownian problem can be approximated 
by a ratio (or tracking) control for the queueing system. While these ob- 
servations are "correct" at a high level, they need to be qualified further. 
Our analysis underscores two sources of approximation errors that need to 
be addressed in order to obtain the refined optimality gaps. First, the value 
function of the Brownian control problem may be substantially different 
than that of the (preemptive) optimal control problem for the queueing sys- 
tem. This difference must be quantified relative to the system size, which 
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we do indirectly through the gradient estimates for the value function of the 
Brownian control problem; this is manifested, for example, in the proof of 
Lemma 6.1. 

The second source of error is in trying to imitate the optimal ratio control 
of the approximating Brownian control by a tracking control in the corre- 
sponding queueing system. The error arises because we insist on having 
a nonpreemptive control for the queueing system. Whereas under a preemp- 
tive control, one may be able to rearrange the queues instantaneously to 
match the tracking function of the Brownian system, this is not possible 
with nonpreemptive controls. Instead, we carefully construct and analyze 
the performance of the proposed nonpreemptive tracking policy. In doing 
so, we prove that the tracking control imitates closely the Brownian system 
with respect to a specific integrated functional of the queueing dynamics 
(see Theorem 5.1 and Remark 5.1). Here too, the gradient estimates for the 
value function of the Brownian system play a key role. 

While the focus of this paper has been a relatively simple model to il- 
lustrate the key ideas behind our approach and the important steps in the 
analysis, we expect that similar results can be established in the cases of 
impatient customers, more general cost structures as well as more general 
network structures. 

As suggested by the preceding analysis, the viability of these extensions 
and others will depend on whether it is possible to (a) solve the sequence of 
Brownian control problems and establish the necessary gradient estimates 
and (b) establish the corresponding approximation result for the nonpre- 
emptive tracking control. 

While we expect that the results of [6] on fully nonlinear elliptic PDEs 
can be invoked for the more general settings, extending our analysis which 
builds on those results may not be always straightforward. In particular, it 
is not immediately obvious how to generalize the proof of the tracking result 
in Theorem 5.1 to more general settings. 

Nevertheless, we can make some observations about the extensions men- 
tioned above: 

• General convex costs. As discussed in the Introduction, the analysis of 
the convex holding cost case will probably be simpler as one tends to 
get "interior" solutions in that case as opposed to the corner solutions 
in the linear cost case, which causes nonsmoothness. We expect that the 
enhanced smoothness (relative to the linear holding cost case) will sim- 
plify the analysis of the HJB equations as well as that of the tracking 
performance. 

• Abandonment. Our starting point in the analysis is that, among preemp- 
tive policies, work conserving policies are optimal. This is not, in general, 
true when customers are impatient and may abandon while waiting (see 
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the discussion in Section 5.1 of [2]). As is the case in [2], our analysis 
will go through also for the case of impatient customers provided that the 
cost structure is such that work conservation is optimal among preemptive 
policies. 
• General networks. Inspired by the generalization of [2], by Atar [1], to 
tree-like networks, we expect, for example, that such a generalization is 
viable in our setting as well. Indeed, we expect that the analysis of the 
(sequence of) HJB equations and the sequence of ADCPs be fairly similar 
for the tree-like network setting. We expect that, in that more general 
setting, it would be more complicated to bound the performance of the 
tracking policies as in Theorem 5.1. 

APPENDIX 

Proof of Lemma 3.1. Up to r", both functions b^{-,u) and a"'{-,u) are 
bounded and Lipschitz continuous (uniformly in u). With these conditions 
satisfied, strong existence and uniqueness follow as in Appendix D of [5]. 
Specifically, strong existence follows by successive approximations as in the 
proof of Theorem 2.9 of [9] and uniqueness follows as in Theorem 2.5 there. 

Proof of Proposition 4.1. Fix k > 0, n £ Z_|_ and a > 0. Recall that (40) 
corresponds to finding 0'^' ^ G C^{B) such that 

(73) = Fa[x, cl>l,ix),Dcl>l,{x),D^cl>lai^)], x€B, 

and so that i?i)"„ = on dB where Fa[-] is as defined in (41). Then, Propo- 
sition 4.1 will follow from Theorem 17.18 in [6] upon verifying certain con- 
ditions. The gradient estimates will also follow from [6] by carefully tracing 
some constants to identify their dependence on k, n and a. 

To that end, note that the function F^{x,z,p,r) [as defined in (42)] is 
linear in the {z,p,r) arguments for all A; G X and x £ B. In turn, this func- 
tion is concave in these arguments. Hence, to apply Theorem 17.18 of [6] 
it remains to establish that condition (17.53) of [6] is satisfied for each of 
these functions. In the following we suppress the constant a > from the 
notation. It suffices to show that there exist constants A < A and t] such 
that uniformly in k £l, y = {x, z,p, r) G F, and ^ G M 

(74) o<A\e\<Y,F,t,[ym<Me, 

(75) ms.x{\F^[y]\,\F^[y]\,\F!:M\AF^Ml\F^M\}<vA, 

(76) maK{\F^[y]\,\F^M\} < ^A(l + |p| + |r|), 
where 

d d 

Fj'j{x,z,p,r) = -—-F''{x,z,p,r), F^^{x,z,p,r) = —F''{x,z,p,r) 
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and 

{F^^{x,z,p,r))iij = -^—^F''{x,z,p,r). 

The other cross-derivatives are defined similarly. We will show that we can 
choose A = eqii, A = ein, rj = 62 for constants eoj^i and S2 that do not de- 
pend on n and a — this will be important in establishing the aforementioned 
gradient estimates. To establish (74) note that, given ^ G M^, 

{l{X'l + fj.i{uin + Xi))S,f, for i=i^i^k, 

\{X^ + ^li{ly,n + x,M-y{e■x), for i=j = k, 

0, otherwise. 
Hence, 

E ^"-^^^i = I E(^^ + ^*('^*™ + ^*))^^' - ^/(^ • ^)^'- 

i,j iaX 

Consequently, for {x,z,r,p) E F we have that 

E ^^MJ ^ ^(^ + /"maxn + /XmaxK\/^fog'" n) ^ ^f + -KV^log™ 72^^ 

where //max = niax^/ifc. In particular, we can choose ei > so that for all 

neZ, 



hj 
To obtain the lower bound note that, for y £T, 

V Ff'jCiCj > 77 (min A," + min^i^V^log™ n) V ef - -e^K^/^log™ n. 

i,j iaX 

Hence, we can find eo > such that for all n. 



Y.Ft^i,i,>eon. 



Note that above £q and £i can depend on k but they do not depend on n 
and a. Hence, we have established (74) and we turn to (75). To that end, 
note that 

Fp Ax,z,p,r) = f{e-x)+lk- HkXk and 
(78) ^ 

Fp^ (x, z,p, r) = l'^ - HiXi for i / k. 

Therefore, 

j 
< IK^/nlog"'n+l+Ima■K{\l'^\ + fiiKy/nlog^n), 
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where we used the simple observation that /(e ■ x) < {e ■ x)'^ + 1. Clearly, 
we can choose 82 so that \Fp\ < e2So^/nlog^ n. Also F^ = —7 and F^x = 
so that by re-choosing 62 large enough we have max{|FJ^[y]|, |F^^, [y]|} < 
e2£o\/nlogn. Finally, by (77) we have that 



pk 

rtjXi 


= 


for i / j, 


pk 

raXj 


= 


for i^k,i^j, 


pk 


1 


for i^k, 



Fruxi = 2^t for i ^ k, 
^Lx, = \f^k-\^J{e-x) 



,k 1 d 

'■'^'"^^ 2 dxk 



<.x, = 7TT7-/(e-^) iorj^k. 



Thus, 



pk |2< 



^^2 

lex 



dxi 



' 1 In 



where we used the fact that /(•) is continuously differentiable with Lipschitz 
constant 1 (independently of a). Finally, 

-^x, = ^/(^ ■x)(ck + fJ^kPk - -^fJ'krkk j - f^iPi + -^f^iru, 

so that 

(79) |F^J < |cfc| +fj.k\p\ + ^fJ.k\r\ +/ii|p| + ^Hi\r\. 

Also, note that 

F.% = ^^/(e-x)(c.+p.-ir,,), 

so that 

pk ^ f 2[cfc + ^fcPfc- iyUfcrfcfc], if|e-x|<i, 
^'^^ i 0, otherwise. 

Combining the above gives 

|F,*^J<e2eo(l + |p| + |r|) 

for suitably redefined £2 which concludes the proof that the conditions (74)- 
(76) hold with A = £in, A = equ and rj = 62- Having verified these conditions. 
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the existence and uniqueness of the solution </>" ^ to (40) now follows from 
Theorem 17.18 in [6]. 

To obtain the gradient estimates in (44) we first outline how the solu- 
tion (f)^ ^ is obtained in [6] as a limit of solutions to smoothed equations (we 
refer the reader to [6], page 466, for the more elaborate description). To that 
end, let F^ be as defined in (42) and for y G T define 

(80) F%] = Gh{Fl[y],...,Fi[y]), 
where 

Gh{y) = h~' j_^^^p[y-^^ Go(y) dy 

and Gq{x) =minjgxXj and p(-) is a mollifier on M^ (see [6], page 466). F^ 
satisfies all the bounds in (74)-(76) uniformly in /i; cf. [6], page 466. Then, 
there exists a unique solution u^ for the equations 

(81) F^[u^] = Q 

on Bl with -u'^ = on dB]!:. 

The solution (/>" ^ is now obtained as a limit of {u^} in the space C^:''^{B) 
as defined in (45). Moreover, since the gradient bounds are shown in [6] to 
be independent of /i, it suffices for our purposes to fix h and focus on the 
construction of the gradient bounds. 

Our starting point is the bound at the bottom of page 461 of [6] by which 

(82) \Ala^B^^<C{a,n){l + \uX^js^), 

where |u'*|2 gn = 'Y1ij=qW^Tj b ^^^^ ['I^bi J = Oi 1'^, are as defined in Section 4. 
The constant a{a,n) depends only on the number of classes / and on A/A 
(see [6], top of page 461) and this fraction equals, in our context, to Ei/sq 
and is thus constant and independent of n and a. 

We will address the constant C{a,n) shortly. We first argue how one 
proceeds from (82). Fix < 5 < 1, let e = 5/C{a,n) and C(e) = 2/(e/8)V" 
(see [6], top of page 132). Then, applying an interpolation inequality (see [6], 
bottom of page 461 and Lemma 6.32 on page 130), it is obtained that 

Plugging this back into (82) one then has 

for a constant C that depends only on 5 and a. In turn, 
for a constant C that does not depend on a or n. 
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Hence, to obtain the required bound in (44) it remains only to 
bound C{a,n). Following [6], building on equation (17.51) of [6], C{a,n) 
is the (minimal) constant that satisfies 

(83) C(l + M2)(l + fiRo + flRl) < C{a,n){l + \u%q), 

where (as stated in [6], bottom of page 460) the (redefined) constant C 
depends only on the number of class / and on A/ A = Ei/eq. The constants fl 
and p, are defined in [6] and we will explicitly define them shortly. Here one 
should not confuse p, with the average service rate in our system. In what 
follows fi will only be used as the constant in [6]. We now bound constants jl 
and jl. These are defined by 

-^ Do _ ^ C{I) (Al Bq 

'^ A(l + M2)' ^ A \Ae I + M2 

Do= sup{\F!^{y,u''{y),Du\y),D\\x))\ 

x,y£B 

+ \F^{y,u\y),Du'^{y),D\\x))\\D^u\y)\}, 

Ao = sup{\F,i\ + \Fl}\}, 
B 

Bo = sup{|Fp,||Z?V| + \F,\\D^u^\ + \F,,\\Du^\ + |F,,|}, 
B 

where C{I) is a constant that depends only on the number of classes /, e G 
(0, 1) is arbitrary and fixed (independent of n and a) and M2 = supg \D'^u^\. 
The constants /Z, ft and M2 are defined in [6], pages 456-460, and ^0 and Bq 
are as on page 461 there. 

We note that F^ is a constant, F^ is bounded by C'-^/nlog'"n for some 

constant C [see (44)] that depends only on k and, by (79), |F^| < £2^0(1 + 
\p\ + \r\). In turn, Dq < 4e2£ov^log"*"'Supg(l + \Du^\ + \D'^u'^\). Arguing 
similarly for Aq and Bq we find that there exists a constant C (that does 
not depend on n and a) such that 

Ao<C^/^log"'n and B^ <Csup{l + \Du''\ + \D^u''\), 

B 

which in turn implies the existence of a redefined constant C such that 
Vra(l + M2) B 



and 



_^Clog^^ C ^ , 2^, 

n n(l + M2) B 
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The proof of the bound is conchided by plugging these back into (83) and 
setting Rq = Hy/nlog"^ n there to get that 

(7(0,71) <Clog^'"(i+i/")n 

for some C that does not depend on a and n. 

The constant C on the right-hand side of (44) (which can depend on n 
but does not depend on a) is argued as in the proof of Theorem 17.17 in [6] 
and we conclude the proof by noting that the global Lipschitz constant (that 
we allow to depend on 77,) follows from Theorem 7.2 in [13]. 

We next turn to proof of Theorem 5.1. First, we will explicitly construct 
the queueing process under the /i-tracking policy and state a lemma that will 
be of use in the proof of the theorem. Define A"(t) = M°'{\^t) so that A" 
is the arrival process of class-i customers. Given a ratio control C/" and 
the associated queueing process X" = (X"',(5",Z",X"), W"^ is as defined 
in (21). Also, we define 

That is, D'^{t) is the total number of service completions by time t in the 
77th system. 

For the construction of the queueing process under the tracking policy 
we define a family of processes {^"^,7 E X, ^ C X} as follows: let {^^;/ G 
Z_|_,/C CX} be a family of i.i.d uniform [0, 1] random variables independent 
of J'oo as defined in (17). For each /C C X, define the processes (A^y^, 7 G X) 
by 

m) A,«(*)=Ll|ivE.,«A,<f'=^lvE,,„Aj- 

We note that for any strict subset % dX and 7 G /C, the probability 
that a jump of D^{t) results in a jump of A^-^ is equal to A"/^^,g^ A^ = 
O'l/Ylken'^k and is strictly greater than Af/^;,gjA^ = Oj. We define 

(85) ej = min Oj - = — , 

and note that e^ > by our assumption that Oj > for all 7 G X (see Sec- 
tion 2). Let e = minjej/4. 

Note that at time intervals in which i G IC{-) = Ti (see Definition 2.1) 
for some / ^ C X, the process A^-^ jumps with probability A"/ Yliki^v. ^k 
whenever a server becomes available (i.e., upon a jump of Z?"). In turn, we 
will use the processes {^"^,7 £l,T-i Cl} to generate (randomized) admis- 
sions to service of class-7 customers under the /i-tracking policy. 

More specifically, under the /i-tracking policy (see Definition 2.1) a cus- 
tomer from the class-7 queue enters service in the following events: 
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(i) A class-i customer that arrives at time t enters service immediately 
if there are idle servers, that is, if (e • X"'(i— ))~ > 0. 

(ii) If a server becomes available at time t (corresponding to a jump 
of D") and t is such that i G IC{t—) =T-L Gl, then a customer from the 
class-i queue is admitted to service at time t with probability A"/^^,g^Afc. 
This admission to service corresponds to a jump of the process -4,"^ as 
defined in (84). 

(iii) If a server becomes available at time t (corresponding to a jump 
of D") and t is such that /C(t-) = and i = min{fc G I:Qfit) > 0}, then 
a class-i customer is admitted to service. 

Formally, the queueing process X" = {X"" , Q'^ , Z'^ , X"") satisfies 

Znt) = ZnO) + f l{(e • X^{s)r > O} dAUs) 
Jo 

+ Y, fl{i&lC{s-)Ms-)=n}dAlnis) 

-Hex ° 

+ / l{IC{s-) = 0,i = mm{k£l:Ql{s-)>O}}dD''{s) 
Jo 

Xrit) = Xr(0) + A^t) - Nt (a^. f^ Z^{s) ds^ ^ G X, 

Q^{t)=X^{t)-Z^{t), ^GX. 

The second, third and fourth terms on the right-hand side of the equation 
for Z" correspond, respectively, to the events described by items (i)-(iii) 
above. Finally, X" is defined from X" as in (8). The fact that the above 
system of equations has a unique solution is proved by induction on arrival 
and service completions times (see, e.g., the proof of Theorem 9.2 of [11]). 
Clearly, X" satisfies (12)-(16) with [/" there constructed from Q"^ using (11). 

We note that, with this construction, the tracking policy is admissible 
in the sense of Definition 2.2. Also, it will be useful for the proof of Theo- 
rem 5.1 to note that with this construction, if [s,t] is an interval such that 
i G K,{u) C X for all u G [s, t] then 

(86) Q^{t) - Q^s) = Ant) - AUs) " E T li^(^-) = ^^ dAl^i^). 

Before proceeding to the proof of Theorem 5.1 the following lemma pro- 
vides preliminary bounds for arbitrary ratio controls. 



40 B. ATA AND I. GURVICH 

Lemma A.l. Fix k,T > and a ratio control V , let X" = (X",Q", 
Z",X") he the associated queueing process and define 

T^T = mf{t > : X"(t) ^ B:} a T log 71. 

Then, there exist constants Ci,C2,Kq > (that depend on T and k hut that 
do not depend on n or on the ratio control U^) such that for all K > Kq 
and all n large enough, 



»| sup \W''{t)\>Ky/^logn} 



(87) 



^0<i<2Tlogn 

P{|X"(t) - X"(s)| > ((t -s) + {t- sf)K^/^logn + Klogn, 

for some s <t < 2Tlogn} 

F{\A^{t) - A^(s) - A^(t -s)\> en{t -s) + Klogn 
(89) for some s<t< t"^} 

< Cie-^2i^'°s", iGX, 



> en{t — s) + Klogn 



p| D^{t) - D^{s) -Y^^ji^v,n{t - s) 

(90) for some s <t< t"^ > 

-4"«(i) - ^Ini^) - Kit -s)< 'jn{t -s)- Klogn 

(91) for some s <t< r"y 

< Cie~^2i^'°s", ieI,}CcI. 

Proof. Equation (87) follows from strong approximations (see, e.g., 
Lemma 2.2. in [4]) and known bounds on the supremum of Brownian mo- 
tion (see, e.g., equation 2.1.53 in [4]). Equation (88) then follows using this 
bound together with (52) in [2] but with X"-{t) -X"(s) instead of X"(i) 
(in the notation of [2] Vt^" is W"^). Equations (89)-(91) follow by carefully 
constructing and bounding the increments. We outline the proof of (89) and 
the others follow similarly. To that end, note that given K and for all n 
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large enough 

{lA^t) - A^s) - Xnt -s)\< ln{t -s) + Klogn, 

for all 0<s<t<2riog7i} 



|^f'"-A['/log'n 



5 < max max — - — ^ — < A'y logn 

U<iVr i>0 : j log' n<3Tlogn Un j j^gi ^ 

where Af^'"" = ^"((i + l)/log'n) - Af (j'/log^n) and iV = max{/:log'n < 
A"/logn}. Indeed, given an interval \s,t) we can construct it from smaller 
intervals. Starting with / = 0, we fit as many intervals of size 1 into [s, i), we 
then continue to fit as many intervals of size 1/ log n to the uncovered part of 
the interval and continue sequentially in /. We omit the simple and detailed 
construction. Note that with such construction, given an interval [s,t), its 
covering uses at most logn intervals of size log n for each / > 0. Also, note 
that N"^ < Clog 71 for all n and some constant C. From here, using strong 
approximations (or bounds for Poisson random variables as in [7]) we have, 
for each j and /, that 

„fl4''"-Af/log'~l ^ ,,^^1 , p^^-c,.,..„. 
I- yjxf/log^n J 

Since the number of intervals considered is of the order of nlogn, the bound 
follows with redefined constants Ci and C2. □ 

Proof of Theorem 5.1. Since k is fixed throughout we use /i"'(-) = /ik"(0- 
As in the statement of the theorem, let 

rix, u) = L{x, u) + Al^lix) - 7</.:(x) for x^Bl.u^U, 

so that by the definition of A^ (x) we have 

^"(x,n) = -70;'(x) 

(92) + (e • x)+ • J^uJq + ^i(0:!)^(x) - ^i^(Oii(x) 



Defining, as before, 

Mf (Z) = Q + iXi{cl>l\{z) - iM*(C)n(^), 

we have that 
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Let [/" be the ratio control associated with the /i"-tracking policy, let X" = 
(X",(5",.Z'",X") be the associated queueing process and define 

^'^(s) = V"(X«(5), [/"(.)) -V"(X"(s),/i"(X"(s))) 

(93) = (e • X"(.))+ Yl K{X-{s))M^{X-{s)) 

-(e.X-(.))+5]C/r(s)Mr(X"(.)). 

iex 

Recall that, by construction, Q'^{s) = (e • X"(s))"''?7f (s) so that (93) can be 
re-written as 

r{s) = rix^{s),u^{s)) - nx^is), k^ix^^is))) 

= (e • X"(.))+ Y. hnX''{s))M^{X-{s)) 



YQns)Mnx''is)). 



J6l 



The theorem will be proved if we show that 



(94) 



E 



e-^^|V'"(s)|ds 



<Clog'="+^n. 



fco+3. 



To that end, define a sequence of times {t"} as follows: 

rr+i = inf{t > rp : /i"(X"(t)) / /i"(X"(rr))} A r^^^j, for I > 0, 
where r^ = r/" A r", j, and 

log"*n 



(95) 



V=to- 



n 



for to = 4K/ej with ej = min-^rxOi ~ ^^ "' as in (85). Finally, we define 

r" = sup{/ G Z+ : r" < r^, ^} and set T^n^i = r", ^. We then have 



',T 



e-^^|V'"(s)|ds 



r"+l n 



1=1 " 'i-i 



1=1 V^^r-1 Jrii^+v" 
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The proof is now divided into three parts. We will show that, under the 
conditions of the theorem, 



(96) 
(97) 



E 



E 



sup sup \^''{s)\ <Clog^°-^^n, 

'-l<Kr"+lT;1j<s<(r,1^+r;")AT," ^ 

sup sup |'(A"(s)|j <Clog'="+2j^, 



where we define sup^^n _,_^„-)<g^^nY(^n ^^„) |V'"'(s)| = if r" < r/^-^ + ??". Fi- 
nally, we will show that 



(98) 



E 



/ 



?7"Ar", 



,T 



\ris)ds\ 



<Clog'=»n 



The proof of (96) hinges on the fact that, sufficiently close to a change 
point T^, all the customer classes, i, for which /i"(X"(s)) = 1 for some s 
in a neighborhood of r", will have similar values of M^(X^(s)). This will 
follow from our gradient estimates for i;^". The proof of (97) hinges on the 
fact that, ?7" time units after a change point r" the queues of all the classes 
for which /i"'(X"'(t")) = are small because, under the tracking policy, these 
classes receive a significant share of the capacity. 

Toward formalizing this intuition, define the following event on the un- 
derlying probability space: 



n{K) = {|X"(t) - X"(s)| < K^log^n{t - s) + Klog 



n, 



for ah s < t < t"^} 
n^cz l^Alnit) - A"h(^) - Kit -s)> |n(t -s)- Klog 



n 



for all s < t < t"^. 

r\,^x{\A^{t) - A'^is) - A,"(i -s)\< en{t -s) + K\ogn 

for all s <t< t'I^^t}- 
For each < t < r ", ^ and z £ X let 

(99) ^rit)=sup{s<t:hnX^is)) = l}, 

(100) C W = infjs > t : /.^(^"(s)) = 1} A r,",,^ 
and 



(101) 



CW 



?f(i) + r/'^, ifQf(t)>4inogn, 
t, otherwise. 
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Then, we claim that on i).{K) and for all t with ?f (t) > ?"(t), 

sup |(e • X^{s))^Uns) - (e • X^ {s))^ h^X^ {s))\ 

.ff(t)<s<?f(t) 

(102) 

< UKlogn. 

Note that since /i"(-) G {0, 1}, the above is equivalently written as 

(103) sup Q^(s)<12Klogn. 

?7(t)<s<?f(t) 

In words, when the process X"(t) enters a region in which hf{X^{-)) = 
the queue of class i will be drained up to 12Eriogn within at most r/" time 
units and it will remain there up to ?f (t). We postpone the proof of (102) 
and use it in proceeding with the proof of the theorem. 
To that end, fix / > and let 

Jl = min arg min Mf (X" (t;" ) ) . 

Then, by the definition of the function /i" in (38) we have that /i^, (X"(t")) = 
1 and hi{X^{Tp)) = for ah i / jf- I^i particular, 

ris) = {e-X-{s))^h],{X^{s))M^.{X-{s)) 

-Y:^Q^{s)M^{X-{s)) 
for ah s^It'I", {tP + r/") A tI\_^). Let 

Then, simple manipulations yield 

\r{s)\ < Yl QUsmnx'^is))] 

(104) +|M|(X"(.)|(e. !"(.))+- Yl Q"(^) 

ieJ(r,")U{jf} 

+ Yl Qns)\MnX^is))-^q{X-{s))\. 

We turn to bound each of the elements on the right-hand side of (104). First, 
note that for ah i ^ JirJ") U {jf} it follows from (103) that 

sup Q^(s)<12Klogn. 

T"<s<(T,"+»;")Ar,'^^ 

Also, by (36) we have for all i G X that 

(105) sup |M;^(X"(s))|<Clog'=in, 



OPTIMALITY GAPS IN THE HALFIN-WHITT REGIME 45 

so that 

(106) Yl Q7(s)|M;"(X"(s))| <12/A'C71og'=i+in 

for all s E [r", (r" + r/") A T;"^|) and a constant C that does not depend on n. 

From (103) and from the fact that ^i^xQ^i^) ~ (^ ' -^""i^))^ ^^ similarly 
have that 



(107) |M'l(X"(s)| 



(e.X-(,))+- J^ Q-(,) 



<127Ci^log'=i+^ 



n. 



To bound the last element on the right-hand side of (104) note that for 
each i G J{tP) there exists tJ" -rf <t< t^ such that hJ^{X'^{t)) = 1. Oth- 
erwise, we would have a contradiction to (102). We now claim that for each 

^Gj(rf), 

(108) |M;^(X"(s))-M]1(X"(s))|<-°^ " 



n 



for all s in [r" — rp ,tJ^ + r/"]. Indeed, by the definition of Q.{K), we have 
that |X"(t)-X"(s)| <Clog'"+2nfor ah s,t in [tJ" -rf^rj" + rf]. As in the 
proof of (37) [see, e.g., (51)] we have that 

(109) \M^{x)-M^{y)\< '^ °g , iEX, 



for x,y £ i3", with |x — y| < Clog'""'' n. In turn, 

(110) |Mf(X"(i))-Mf(X"(s))|<— ^ n Clog n 



n \/n 



for all z G X and all s,t £ [r" — r]'^,Tp + r/"] where we used the fact that 
ki = k2 + m. Since, for each j G i7(t") , there exists tJ^ — rp <t< Ti such that 
/i^(X"(t)) = 1 we have, by the definition of /i" that j £ argmini6xM;"(X"(t)) 
for such t so that (108) now follows from (110). Finally, recall that 
EiGxQ"(0 = (e-^"(s))+ < K^^log™ n for all s < r^, .^ and that ko = ki+m 
so that by (108) 

^ Qns)|Mf(X"(s)) -M;1(X"(s))| <Clog^«+2^. 

Plugging this into (104) together with (106) and (107) we then have that, 
on n{K), 

sup |V'"(s)| <Clog'=i+'"+2n = Cinog'=°+2^. 
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This argument is repeated for each /. To complete the proof of (96) note that, 
using (105) together with supo<s<T-n \e-X"'{s)\ < K,^/nlog"^ n, we have that 

supo<s<r" |V'"('5)| < Cy/nlog "^~^^n. Applying Holder's inequality we have 
that 



E 



sup sup l''/'"'('5)| 

l</<r"+lT,"_^<s<(T,"_j+»7") 



<E 



sup sup \'il^''{s)\l{n{K)} 

l<l<r"+lT;i^<s<{Tp_j^+rj") 



+ E 



max sup \r{s)\m{Kr} 

l<Kr.n+l rj"_^<s<(T;1^+r?") 



< Clog'^0+2 ^ + CV^log'=i+'" nCie-(^2^/2)iogn 

for redefined constants Ci,C2 and (96) now follows by choosing K large 
enough. 

We turn to prove (97). Rearranging terms in (93) we write 

r{s) = ^ Mr(X"(s))((e • X-{s))^hnX-is)) - (e • X^{s))'-Uns)), 

so that equation (97) now follows directly from (102) and (105) through an 
application of Holder's inequality. 

Finally, to establish (98), note that from the definition of r", j., 

sup \ris)\<i sup |i"(t)i^Mr(x"(t)) 



0<t<)7"AT^, y 



0<t<r)"Ar^, J, 



</ sup Clog''^n\X''{t)\<CK^/K\og 



ki+m 



n. 



0<t<ri"AT^, J, 



In turn, 
(111) 



E 



-7t|J,n 



\rit)\dt 



<Clog 



ki+m 



n ■■ 



Clog^^o 



n. 



We have thus proved (96)-(98) and to conclude the proof of the theorem it 
remains only to establish (102). To that end, let <f (i), <?"(*) and ^f{t) be as 
in (99)-(101). Fix an interval [l,s)e (?r(t),'ff (t)) such that Qf('u) > 2K\ogn 
for all u £ [/,s). By the definition of the tracking policy, (86) holds on this 
interval so that, on tj € Q.{K), 



U 



(112) 



Q?(0 - Ql{s) < en{t - s)n - -^n{t - s) + 2K\ogn 
<-jn{t-s) + 2K\ogn. 
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Equation (102) now follows directly from (112). Indeed, note for all t < t^'^t, 
Q[^(t) < (e-X"(t))+ < |X"(t)| < K^^log^n. Hence, Q^Ut)) < n^log'^'n. 
In turn, using (112) and assuming that ^l^{t) > ?f'(i) + ??" we have that 
Qii'^Oii^)) — ^Klogn for some time ?o i(*) — '^i'i'^) + ^" ^^t^ vT as defined 
in (95). Also, let 

?£,(t) = inf{t > ,o",,(i) : Q?(t) > 12i^logn} 

and 

?M(i) = sup{t < <j£,(i) : Qr(t) < SKlogn}. 

Note that (112) applies to any subinterval [/,«) of [?"j(i),^2^j(i))- I'^ turn, 
"?2'i(*) — ?r(*) would constitute a contradiction to (112) so that we must 
have that Q'l{s) < 12Klogn for ah s e [?o,» (*)>?"(*) ) with ^^^^(t) < ?"(t) + r/". 
Finally, note that ?J'j(t) can be taken to be t if Q"(t) < 4iClogn. 

This concludes the proof of (102) and, in turn, the proof of the theorem. 



Proof of Lemma 6.2. Let T, r", rp and (x",g'^) be as in the statement of 
the lemma. We first prove (59). To that end, we claim that, for all T large 
enough, 



(113) 



^x",q" 



Tlogn 



e-^^(e-c)(e-X"(s))+ds 



<Clog2n 



for some C > and all n E Z. This is a direct consequence of Lemma 3 in [2] 
that, in our notation, guarantees that 

Ex",g" [|^"(i)|] < C{1 + |x"| + V^(t + t^)) 

for all t > and some constant C > 0. We use (113) to prove Lemma 6.2. 
The assertion of the lemma will be established by showing that 

r-2Tlogn 

e-^^(e-c)(e-X"(s))+(is 



E, 



x'%q" 



<Clog'n. 



To that end, applying Holder's inequality, we have 

c2T\ogn 

e-^^(e-c)(e-X"(s))+ds 



E, 



■x'\q" 



(114) 



^ Jlij'*;^ nf^ 



(2riogn-r^,y)+ sup (e • c)(e • X'^(t))+ds 

0<t<2Tlogn 



< jE,.,,„[((2riogn-r;,^)+)"] 



IE. 



•x"-,q" 



sup {e-c){e-X'^{t)yds 

^0<t<2Tlogn 
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Using Lemma A.l we have that 



(115) 



^x",q" 



sup {e-c){e-X''{t)rds 

'-^0<t<2Tlogn 



< Cn log n 



for some C > (that can depend on T). Also, since tti > 3, 

F{Tj!,j,<2T\ogn}<F\ sup {X"- {t)\ > k ^nlog^ n - M ^} . 

Choosing k' (and in turn k. large enough) we then have, using Lemma A.l, 
that 

P{r:,^<2riogn}<-^ 



n^ 



(116) 

and hence, that 

(117) E,n,,„[((2riogn-T:,,r)+)'] <C- 

Plugging (115) and (117) into (114) we then have that 



(118) 



IEa;",q" 



2Tlos,n 



e-^^(e-c)(e-X"(s))+(is 



<Clog^n. 



To conclude the proof we will show that (60) follows from our analysis 
thus far. Indeed, 



E\e 



-7V,T^n(X"(r:,,^))] 

f2riogn 






-•ys 



sup (e-c)(e-X"(s))+(is 



k',T 



0<s<2Tlogn 



The right-hand side here is bounded by C log n by the same argument that 
leads to (118). 

Proof of Lemma 6.3. Recall that W" is defined by Wp{t) = M^^^{t) - 
M^^i*), where 



-I,, {X^{s)-U:{s){e-X-{s)r)ds. 
Jo 

The fact that each of the processes Mp^{t) and M"2(i) are square integrable 
martingales with respect to the filtration (-F") follows as in Section 3 of [12] 
and specifically as in Lemma 3.2 there. 
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Since, with probability 1, there are no simultaneous jumps oiN"" and A/"/, 
the quadratic variation process satisfies 



[wnt = mi]t + wu 



s<t s<t 

where the last equality follows again from Lemma 3.1 in [12] (see also Exam- 
ple 5.65 in [14]). Finally, the predictable quadratic variation process satisfies 

{Wnt = (M-i)t + (M-2), 

= X'^t + fii [ (Xf (s) + i^^n - Ul'{s){e ■ X"(s))+) ds 
Jo 

\anX^{s),U-is))fds, 



where the second equality follow again follows from Lemma 3.1 in [12] and 
the last equality from the definition of o"f(-,-) [see (25)]. By Theorem 3.2 
in [12] {iW^{t)f - [W[']t,t> 0]) and ((^^"(t))^ - [Wi"]t,t > 0) are both mar- 
tingales with respect to {J~t)- In turn, by the optional stopping theorem so 
are the processes A^^(-) and Vf (•) as defined in the statement of the lemma. 
Finally, it is easy to verify that these are square integrable martingales using 
the fact the time changes are bounded for all finite t. 
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